Increasingly, client-side input or output (I/O) caches are being employed on compute servers in order to decrease I/O latency to applications or to offload work from the expensive networked storage appliance to cheaper commodity hardware. The cost/performance ratio of this solution has two competing considerations as discussed below.
On one hand the storage media, e.g. flash memory, used to implement the I/O cache on the compute server is cheaper than the same cache-storage media in the storage appliance. It is also faster to access a cache on the computer server than on the storage appliance, since the network latency is eliminated. As a result, this argues for moving cache from the storage appliance where the cache is shared by all clients to the compute server where it is used locally to that server.
On the other hand, a set of isolated caches that do not share their cache storage will not have as high a hit rate as a single cache that combines the storage of all of them. This assumes that the working set sizes vary dynamically across the compute servers, so that at any given time, some of them are under-provisioned with client-side cache and some are over-provisioned resulting in inefficiencies.
Further with prior existing technology, attempts to manage cache have considered the use of a complex distributed data structure with elements spread across the cache cluster. Unfortunately, to maintain this complex distributed data structure would require one or more network messages each time an object in any cache was accessed which would be cumbersome and inefficient.